As the Internet has matured, the format characteristics of the content available on the Internet have changed. Sound and video content is now mixed in with the traditional textual content. However, this new content on the Internet requires a greater connection speed (i.e., bandwidth) than was commonly available a few years ago.
FIG. 1 illustrates an example of a typical Internet configuration. It includes a server (such as media server 20), which is coupled to the Internet 30. The server typically includes one or more physical server computers 22 with one or more physical storage devices and/or databases 24. On the other side of an Internet transmission is a client 90, which is connected via one of many available Internet Service Providers (ISPs) 80. Herein, a server is a network entity that sends data and a client is a network entity that receives data.
Cloud 30 is labeled the Internet, but it is understood that this cloud represents that portion of the Internet that does not include the server, client's ISP, and the client. Inside such cloud are the routers, transmission lines, connections, and other devices that more-often-than-not successfully transmit data between clients and servers. Inside exemplary Internet cloud 30 are routers 32–44; two satellite dishes 46 and 50; and a satellite 48. These represent the possible paths that a data packet may take on its way between the server and the client.
Bandwidth
Bandwidth is the amount of data that can be transmitted in a fixed amount of time. For example, bandwidth between media server 20 in FIG. 1 to media client 90 is calculated by the amount of data (e.g., 1000 bits) that may be transmitted between them in a unit of time (e.g., one second).
As shown in FIG. 1, a transmission over the Internet travels across multiple links before it reaches its destination. Each link has its own bandwidth. Like a chain being only as strong as its weakest link, the maximum bandwidth between server 20 and client 90 is the link therebetween with the slowest bandwidth. Typically, that is the link (such as link 82 in FIG. 1) between the client 90 and its ISPs 80. That slowest bandwidth is the maximum de facto bandwidth.
Herein, unless otherwise apparent from the context, references to bandwidth between network entities (such as server 20 and client 90) is assumed to be the maximum de facto bandwidth therebetween.
Bandwidth may also be called “connection speed”, “speed”, or “rate”. In references to bandwidth measured by bits per second, it may also be called “bit rate” or “bitrate.”
Streaming Media
Streaming is a technique for transferring multimedia data such that it can be processed as a steady and continuous stream. Streaming technologies are becoming increasingly important with the growth of the Internet because most users do not have fast enough access to download large multimedia files quickly. With streaming, the client browser or plug-in can start displaying the data before the entire file has been transmitted.
For streaming to work, the client side receiving the data must be able to collect the data and send it as a steady stream to the application that is processing the data and converting it to sound or pictures. This means that if the streaming client receives the data more quickly than required, it needs to save the excess data in a buffer. If the data doesn't come quickly enough, however, the presentation of the data will not be smooth.
Within the context of an audio and/or visual presentation, “media” and “multimedia” are used interchangeably herein. Media refers to the presentation of text, graphics, video, animation, and/or sound in an integrated way.
“Streaming media” is an audio and/or visual presentation that is transmitted over a network (such as the Internet) to an end-user. Such transmission is performed so that the presentation is relatively smooth and not jerky. Long pauses while additional frames are being downloaded to the user are annoying to the user. These annoyances encourage a user to avoid viewing future streaming media.
Smoothly Transmitting Streaming Media
Since the bandwidth determines the rate at which the client will receive data, a streaming media presentation may only be presented at a rate no greater than what the bandwidth allows. For example, assume media server 20 needs to send data at 50 Kbps to the client 90 in order to smoothly “play” a streaming media presentation. However, the bandwidth between the client and server is only 30 Kbps. The result is a jerky and jumpy media presentation.
In an effort to alleviate this problem, streaming media presentations are often encoded into multiple formats with differing degrees of qualities. The formats with the lowest quality (e.g., small size, low resolution, small color palette) have the least amount of data to push to the client over a given time. Therefore, a client over a slow link can smoothly present the streaming media presentation, but the quality of the presentation suffers. The formats with the highest quality (e.g., full screen size, high resolution, large color palette) have the greatest amount of data to push to the client over a given time. Therefore, the client with a fast link can smoothly present the streaming media presentation and still provide a high quality presentation.
Select-a-Bandwidth Approach
When a server sends streaming media to a client, it needs to know what format to use. Thus, in order to select the proper format, the server must to know the bandwidth between the server and the client.
This easiest way to accomplish this is to ask the user of the client what their bandwidth is. Since a client's link to the Internet is typically the bandwidth bottleneck, knowing the bandwidth of this link typically indicates the actual bandwidth.
FIG. 2 shows a cut-away 100 of a Web page displayed on a client's computer. Inside the cut-away 100, is a typical user-interface 110 that may be used to ask a user what their connection speed is. The user clicks on one of the three buttons 112, 114, and 116 provided by the user-interface 110. If the user clicks on button 112, the server delivers data from a file containing streaming media in a format designed for transmission at 28.8 Kbps. Likewise, if the user clicks on button 114, data sends from a file containing streaming media in a format designed for transmission at 56.6 Kbps. If the user clicks on button 114, the server delivers data from a file containing streaming media in a format designed for transmission at a rate greater than 56.6 Kbps and up-to the typical speed of a T1 connection.
However, the primary problem with the “select-a-bandwidth” approach is that it requires a thoughtful selection by a user. This approach invites selection errors.
It requires that a user care, understand, and have knowledge of her connection speed. Often, a user does not pay particular attention to which button to press. The user may only know that a media presentation will appear if the user presses one of these buttons. Therefore, they press any one of them.
Often, a user does not understand the concept of bandwidth. A user may choose button 116 because she may want to see the presentation at its highest quality. This user does not realize that seeing the presentation at its highest quality may result in a non-smooth presentation because her Internet connection cannot handle the rate that the data is being sent through it.
If she does understand the concept of bandwidth, then the user may not know her bandwidth. A user may simply be ignorant of her bandwidth. In addition, varying degrees of noise may cause varying connection speeds each time a user connects to the Internet. Furthermore, some types of connections (such as a cable modem) can have wide degrees of connection speed depending upon numerous factors.
Moreover, the user needs to understand the implications of an incorrect choice. A user needs to be educated so that she understands that she needs to select an option that is equal to or less than her bandwidth to get a smooth presentation. But she should not choose one that is significantly less than her bandwidth. If she does, then she will be seeing a smooth presentation at a lower quality that she could otherwise see at a higher available bandwidth.
As can be seen by the above discussion, this manual approach is often confusing and intimidating to many user. Therefore, it often results in incorrect selections.
What's more, maintaining multiple files (one for each bandwidth) at the media server adds to the overhead of maintaining a Web site.
Automatic Bandwidth Detection
To overcome these problems, media servers use a single file containing subfiles for multiple bandwidths. Also, media servers automatically detect the bandwidth.
This single file is called a MBR (multiple bit rate) file. The MBR files typically include multiple differing “bands” or “streams.” These bands may be called “subfiles.” A user only clicks on one link. Automatically, behind the scenes, the server determines the proper stream to send to the client based on the speed selected by the client.
In an environment where end-to-end latency is very high, this automatic speed detection may take a long time. This means that an additional five to thirty seconds is added to the user's wait for the presentation to begin. One factor in this delay for existing automatic speed detection is because of long “handshaking” times while the speed determination is going on.
One existing automatic detection technique involves sending multiple data packets for measuring the speed between the server and client. This technique is described further below in the section titled, “Multiple Measurement Packets Technique.”
Bandwidth Measurement Packets
Typically, automatic bandwidth detection techniques measure bandwidth between entities on a network by sending one or more packets of a known size.
FIG. 3 shows a time graph tracking the transmission of two such packets (Px and Py) between a sender (e.g., server) and a receiver (e.g., client). The server and client sides are labeled so. On the graph, time advanced downwardly.
Time ta indicates the time at the server the transmission of Px begins. Time tb indicates the time at the server the transmission of Px ends. Similarly, Time t0 indicates the time at the client begins receiving Px. Time t1, indicates the time at the client completes reception of Px. At t1, the network hardware presumably passes the packet up the communication layers to the application layer.
Packet Py is similarly labeled on the time graph of FIG. 3. tc is the server time at the transmission of Py begins. td is the server time that the transmission of Py ends. Similarly, t2 the client time that it begins receiving Py. t3 is the client time that it completes reception of Py. At t3, the network hardware presumably passes the packet up the communication layers to the application layer.
Bandwidth Measurement Using a Single Packet.
In a controlled, laboratory-like environment, measuring bandwidth between two entities on a network is straightforward. To make such a calculation, send a packet of a known size from one entity to the other and measure the transmission latency, which is the amount of time it takes a packet to travel from source to destination. Given this scenario, one must know the time that the packet was sent and the time that the packet arrived.
This technique is nearly completely impractical outside of the laboratory setting. It cannot be used in an asynchronous network (like the Internet) because it requires synchronization between the client and server. Both must be using the same clock.
Alternatively, the client may track the time it begins receiving a packet (such as t0 for Px) and the time the packet is completely received (such as t1 for Px).
FIG. 3 shows packet Px being sent from a server to a client. Px has a known size in bits of PS. The formula for calculating bandwidth (bw) is
                              bw          ⁡                      (                          P              x                        )                          =                  PS                                    t              1                        -                          t              0                                                          Formula        ⁢                                  ⁢        1        ⁢                                  ⁢                  (                      Single            ⁢                                                  ⁢            Packet                    )                    
This technique works in theory, but unfortunately does not work in practice. Only the hardware knows when a packet is initially received. Therefore, only the hardware knows when t0 is.
The other communication layers (such as the transport layer and the application layer) can only discover the time when the packet is completely received by the hardware. That is when the hardware passes it up to them. This completion time for packet Px is t1. It is not possible to calculate bandwidth only one knowing one point in time.
Packet-pair. A technique called packet-pair is used to overcome these problems in asynchronous networks. With packet-pair, the server sends a pair of packets, one immediately after the other. The bandwidth is determined by dividing the packet size by the time difference in reception of each packet.
Each packet has specific measurable characteristics. In particular, these characteristics include its packet size (PS) and the measured time such a packet arrives (e.g., t0-3 in FIG. 3). Some characteristics (such as packet size) may be specified rather than measured, but they may be measured if so desired.
As shown in FIG. 3, the server sends packet, Px. The client's hardware begins receiving the packet at t0. When reception of the packet is complete at t1, the hardware passes it up the communication layers. Ultimately, it is received by the destination layer (e.g., application layer) at presumably t1.
After the server sends Px (which is completed at tb), it immediately sends packet Py at tc. It is important that there be either 1) absolutely no measurable delay between tb and tc or 2) a delay of a known length between tb and tc. Herein, to simplify the description, it will be assumed that there is no measurable delay between tb and tc.
The client's hardware begins receiving Py at t2. When reception of the packet is complete at t3, the hardware passes it up the communication layers. Ultimately, it is received by the destination layer (e.g., application layer) at presumably t3.
FIG. 3 shows no delay between t1 (the time of completion of reception of Px) and t2 (the time reception of Py begins). Theoretically, this will always be the case if Px and Py are transmitted under identical conditions. In practice, is the often the case because Py is sent immediately after Px.
Using packet-pair, the formula for calculating bandwidth (bw) is
                              bw          ⁡                      (                                          P                x                            ⁢                              P                y                                      )                          =                  PS                                    t              3                        -                          t              1                                                          Formula        ⁢                                  ⁢        2        ⁢                                  ⁢                  (                      Packet            ⁢                          -                        ⁢            Pair                    )                    
This technique works in theory and in practice. However, it only works well over a network that is relatively static.
For example, in FIG. 1, assume the network consists of only the server 20; routers 32, 34, and 36; a specific ISP of ISPs 80; and client 90. Further, assume that the links between each node on this static network is fixed and has a consistent bandwidth. In this situation, the packet-pair techniques provide an accurate and effective measurement of bandwidth.
Packet-pair does not work well over the Internet. However, the packet-pair technique does not work well over a dynamic network, like the Internet. A dynamic network is one where there is a possibility that a packet may be handled in a manner different from an earlier packet or different from a later packet.
FIG. 1 illustrates examples of those handling differences. Assume that all packets are traveling from the server to the client (from left to right in FIG. 1). Assume that packets 60–68 were sent back-to-back by the server 20 to the client 90. Assume that packet 70 was sent by another server (not shown) to the client 90 and it is unrelated to bandwidth measurement.
Notice, as illustrated in FIG. 1, that packets may take different routes. In addition, some routes may significantly delay the packet transmission. This is especially true if the packet is transmitted via an apparently unusual (but not necessarily uncommon) route, such as wireless transmission, oversees via an underwater cable, satellite transmission (as shown by dishes 46 and 50 and satellite 48), etc.
A router (such as router 42) may delay a packet (such as 64) more than another may by temporarily buffering it. Another packet (such as packet 70) from another source may slip in between two packets (such as packets 60 and 62). In addition, a modem (not shown) of the client may compress packets.
Communications equipment (such as a modem) may compress a packet (such as 66) to shrink the packet size and thus speed along transmission. Such packet compression can significantly affect the bandwidth measurement because not all of the subsequent data packets will be compressed or compressed at the same rate.
Multiple Measurement Packets Technique
To overcome these problems, conventional automatic bandwidth measurement techniques uses multiple packets. A server sends several (much more than two) packets and calculates the speed of each. Conventional wisdom on bandwidth measurement indicates that in order to get accurate measurements several pairs of packets must be sent repeatedly over several seconds to several minutes. Herein, this technique is called “multiple-packets” to distinguish it from the above-described “packet-pair” technique.
Typically, the ultimate bandwidth is determined by finding the average of the many bandwidth measurements. This averaging smoothes out variances in delays for each packet; however, it does not compensate for packet compression during transmission. One of two extremely incorrect measurements will skew the average.
Unfortunately, this technique takes a long time relative the existing wait for the user between click and media presentation. A long time may be five seconds to several minutes depending on the data and the situation. Such a delay adds to the annoyance factor for the user who wishes experience the media presentation. This is not an acceptable delay. Since there are no other options available using conventional techniques, the user is forced to endure these delays.
Moreover, these conventional approaches typically use TCP to transmit the packets. Using TCP introduces additional delays for handshaking. These conventional approaches typically modify the kernel of the operating system (usually the transport layer) to perform these measurements.
Varying Bandwidth
Another problem encountered with streaming multimedia content is that, after the initial bandwidth measurement is taken and a streaming rate is determined, factors that influenced the selection of the streaming rate may change. As a result, the bandwidth that is available for streaming media changes as well. For example, if network congestion was a problem at the time the measurement was taken, then the selected streaming rate may be lower than it could be when the network congestion clears. Conversely, if network congestion occurs after the streaming rate has been selected, then the quality of the multimedia presentation may suffer as a result of streaming at a faster rate than the network can accommodate.
Sole reliance on initial bandwidth measurements, therefore, may provide a less than optimum streaming experience.